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ABSTRACT 

The Connecticut Mastery Test is a critical element of 
the state's efforts to attain educational equity and excellence. The 
testing program assesses essential skills in mathematics and language 
arts, including listening, reading, and writing for grades 4, 6, and 
8. Student achievement is measured and reported in relation to 
;jpecific learning objectives (criterion-referenced) that students can 
reasonably be expected to have mastered by the end of grades 3, 5, 
and 7. This report provides an overview and summary of the 
implementation of the grade-6 Connecticut mastery test. In 1991, 
sixth graders mastered an average 24.7 of the 36 mathematics 
objectives tested, a slight increase above 1990, and 84.2 percent 
scored at or above the remedial standard. In language arts, sixth 
graders mastered an average of 8.1 of 11 objectives tested, 
representing no change from last year. There were slight improvements 
in the writing scores, the number of students scoring above the 
remedial standard in writing, and the number of students scoring 
above the writing goal. Reading achievement on the Degrees of Reading 
Power (an average of 57 units) was unchanged from 1990. A comparison 
of results from 1986 through 1991 is provided. Fourteen charts 
present test results and comparisons. Twelve appendixes provide 
supplemental information about testing and scoring. (SLD) 
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LEGISUTIVE BAOCGROUND 



In June 1984, the General Assembly of the State of Connecticut amended Section 
10-14 m-r of the Connecticut General Statutes, an act concerning Education 
Evaluation and Remedial Assistance (EERA)» This law provides that: 

o By May 1, 1985, each local or regional board of education shall have 
developed and submitted for State Board of Education approval, a new 
plan of educational evaluation and remedial assistance. Each plan 
had to address the following: 

0 the use of student assessment results for instructional 
improvement; 

0 the identification of individual students in need of remedial 
assistance in language arts/reading and mathematics; 

0 the provision of remedial assistance to students with identified 
needs; and 

o the evaluation of the effectiveness of the instructional 
programs in language arts/reading and mathematics. 

0 The State Board of Education shall administer an annual statewide 

mastery test in language arts/reading and mathematics to all fourth-, 
sixth- and eighth-grade students, with the following exceptions: 

0 Special Education students who are excluded by a Planning and 
Placement Team (PPT) decision; 

0 students who have been enrolled in an "English as a Second 
Language" program for two years or less; or 

0 students enrolled in a Bilingual Program (as defined in Section 
10-17e of the Connecticut General Statutes) for two years or 
less. 

0 Each student who scores below the statewide remedial standard on one 
or more parts of the eighth-grade mastery examination shall be 
retested. These students shall be retested annually, using the 
eighth-grade mastery test, only in the deficient area(s) until such 
students score at or above the statewide remedial standard(s), 

0 Biennially, each local or regional board of education shall submit to 
the State Board of Education a report which includes indicators of 
student achievement and instructional improvement. 

0 On a regularly scheduled basis, the State Board of Education shall 
complete field assessments of the implementation of local EERA plans. 



0 On an annual basis, test results and low income data shall be used to 
determine the distribution of available state funds to support 
remedial assistance programs. 

The purpose of this report is to provide an overview and summery of the 
implementation of the sixth-grade Connecticut Mastery Test. The mastery test 
assesses how well each student is performing on those skills identified by 
content experts and practicing educators as important for students entering 
sixth grade to have mastered. 
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FOREWORD 



The Connecticut Mastery Test is a critical element in Connecticut's agenda to 
attain educational equity and excellence. The testing program assesses 
essential skills in mathematics and language arts, including listening, 
reading and writing, for grades four, six and eight students. Student 
achievement is measured and reported in relation to specific learning 
objectives that students reasonably can be expected to have mastered by the 
end of grades three, five and seven. 

The Connecticut Mastery Test provides valuable educational information which 
can be used to improve instruction and elevate the achievement of 
Connecticut's students. The test results are reported in a manner that 
identifies how well each student is succeeding in relation to clearly defined 
and meaningful standards. It is our hope that educators throughout the state 
use the results as a tool to gain better understanding of the learning 
occurring in our classrooms and the ways to increase learning in the future. 

Connecticut is committed to an annual cycle of assessment in order to promote: 

0 the monitoring of individual student achievement; 

0 the evaluation of instructional program effectiveness; 

0 educational goal setting; and 

o remedial assistance program improvement. 

An examination of the results since 1985 reveals many signs of steady, 
incremental improvement. The general improvement since the start of the 
program is quite impressive in some areas. Yet the many Connecticut educators 
who helped to build the program had the foresight to include some very 
demanding content and standards. Student performance in relation to these 
expectations reveals that much remains to be done. 

As you examine these results, it is our hope that the many stories they tell 
will prove useful and informative. Department staff are available to 
facilitate the interpretations and application of these test scores. 




Peter Behuniak 
Acting Chief 

Bureau of Evaluation and Student Assessment 
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OVERVIEW OF THE MASTERY TESTING PROGRAM 



In the spring of 1984, the Connecticut General Assembly amended the Education 
Evaluation and Remedial Assistance (EERA) legis'^ation to authorize the 
creation of mastery tests in the basic skill areas of mathematics and language 
arts, including listening, reading and writing skills* The tests were to be 
established for grades four, six and eight ♦ 

The goals of the mastery testing program are: 

0 earlier identification of students needing remedial education; 

0 testing a more comprehensive range of academic skills; 

0 setting high expectations and standards for student achievement; 

0 more useful test achievement information about students, schools and 
districts; 

0 improved assessment of suitable equal educational opportunities; and 

0 continual monitoring of students in grades four, six and eight. 

The type of test that best addresses these goals is a criterion-referenced 
test* Criterion-referenced tests are designed to assess the specific skill 
levels of students. Such tests usually cover relatively small units of 
content. Their scores have meaning in terms of what each student knows or can 
do. Test results are used to identify the areas of strengths and weaknesses 
of each student. 

MASTERY TEST CONTENT 

The CMT 1s designed to assess essent:ial language arts/reading, writing and 
mathematics skills that can reasonably be expected to be mastered by most 
students by the end of the third, fifth and seventh grades. The specific 
skills to be tested within these content areas were identified by committees 
of educators from throughout the state. In addition, surveys were sent to 
many teachers, administrators and parents to determine the appropriateness of 
these skills for the Mastery Test. A complete description of the procedures 
used in the development of the sixth-grade CMT can be found in Appendix A 
(p. 33). 

Mathematics 

The Mathematics Advisory Committee recommended a grade six mathematics test 
that assessed thirty-six (36) specific objectives in four domains: 
(1) Conceptual Understanding; (2) Computational Skills; (3) Problem 
Solving/Applications; and (4) Measurement/Geometry. There are four test items 
per objective for a total of 144 items on the mathematics test. A detailed 
list of domains and objectives is given in Appendix B (p. 37). 
Beginning with the fall 1990 administration of the grade six test, two 
objectives were deleted and two new objectives were added. At the request of 
the Mathematics Advisory Committee and in response to concerns about the 
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developmental appropriateness of requiring mastery of abstract fractional 
computation early in grade six, "adding fractions with like denominators, 
requiring regrouping," and "adding and subtracting fractions with unlike 
denominators" were replaced with one conceptual understanding objective 
involving "conversion between mixed numbers and improper fractions" and one 
computational skills objective involving "estimation of sums and differences 
of fractions and mixed numbers •" 



Language Arts 

The Language Arts Advisory Committee recommended a 112-item grade six language 
arts test that covers two domains: Reading/Listening and Writing/Study 
Skills. Eleven (11) objectives were recommended by the Language Arts Advisory 
Committee. 

The general content of Reading/Listening consisted of narrative, expository 
and persuasive passages on a variety of topics measuring a student's ability 
in: (1) Literal Comprehension; (2) Inferential Comprehension; and (3) 
Evaluative Comprehension. Audiotapes were used to assess students' listening 
comprehension ability in: (1) Literal Comprehension and (2) Inferential and 
Evaluative Comprehension. The Degrees of Reading Power (DRP) test was also 
used to assess reading. The DRP test included eleven (11) passages and 
seventy-seven (77) test items. It was designed to measure a student's ability 
to understand nonfiction English prose at different levels of reading 
difficulty. 

The general content area of Writing/Study Skills consisted of three 
components. First, there was a writing sample for direct, holistic assessment 
of student writing. Each student was asked to write a composition on a 
designated topic. Writing was then judged on a student's demonstrated ability 
to convey information in a coherent and organized fashion. Second, the 
mechanics of good writing, which was defined as (1) Capitalization and 
Punctuation, (2) Spelling, Homonyms and Abbreviations, (3) Agreement and 
(4) Tone, was assessed in a multiple-choice format. Third, Study Skills was 
assessed through Locating Information and Note-taking/0u1:lining. Locating 
Information (Schedules, Maps, Index and Reference Use and Dictionary Meaning) 
measured students' ability to find ano use information from the sources 
listed. Note-taking and Outlining tested a student's ability to take notes 
and report information as well as complete missing outline information. A 
detailed list with objectives and number of items per objective is given in 
Appendix C (p. 41). 

FUTURE DEVELOPMENT 

The Connecticut State Department of Education, in conjunction with content 
consultants and various CMT advisory committees, has begun the development of 
the second generation of the CMT. The current CMT is under review to 
determine which skills are appropriate for inclusion on the new test. In 
addition, new content areas and other forms of assessment techniques (e.g., 
performance assessment and short-answer questions) are being considered. It 
is anticipated that the second generation CMT will be administered for the 
first time statewide in the fall of 1993. Items for this set of exams were 
piloted in the fall of 1991 and will be followed by a second pilot in the fall 
of 1992. 
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SEHING MASTERY STANDARDS BY OBJECTIVE 



The essence of the Connecticut Mastery Test (CMT) is the establishment of a 
specific mastery standard against which each student's knowledge and competency 
on each objective can be compared. The mastery test incorporates appropriate and 
challenging expectations for Connecticut public school students. The goal of the 
CMT Program is for each student to achieve mastery of all objectives. The 
objectives being tested were identified as appropriate and reasonable for 
students at each of the grades tested. These tests are designed to measure a 
student's performance on these specific objectives. 

The process of establishing the mastery standards by objective used a statistical 
method that required two decisions to be utilized. The first decision defined a 
student who mastered a particular skill as one who had a 95% chance of correctly 
answering each item within the objective. The second decision was that the 
specific standard for each objective would identify 99X of the students who 
mastered the skill. By applying the two decision rules st^^.ted above to a 
binomial distribution table, mastery standards were established for the 36 
mathematics objectives and the 11 language arts objectives. 

The mastery standards are as follows: 

0 In mathematics, for each of the 36 objectives, a student must answer 
correctly at least 3 out of 4 items. 

n In language arts, for the 11 multiple-choice objectives with varying 
numbers of items, a student must answer correctly the following numbers 



of items: 



WRITING MECHANICS 



# Items Correct 
for Mastery 



(1) Capitalization & Punctuation 


9 


out 


of 


12 


(2) Spelling 


7 


out 


of 


9 


(3) Agreement 


11 


out 


of 


15 


(4) Tone 


3 


out 


of 


4 


STUDY SKILLS 










(5) Locating Information 


8 


out 


of 


11 


(6) Note-taking and Outlining 


3 


out 


of 


5 


LISTENING COMPREHENSION 










(7) Literal 


4 


out 


of 


6 


(8) Inferential and Evaluative 


10 


out 


of 


14 


READING COMPREHENSION 










(9) Literal 


6 


out 


of 


8 


(10) Inferential 


10 


out 


of 


14 


(11) Evaluative 


10 


out 


of 


14 



No mastery standards were set for the two holistic language arts measures, 
neither the Degrees of Reading Power (DRP) test nor the Writing Sample, since 
these measures are not composed of objectives on which mastery could be assessed, 
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SETTING REMEDIAL (GRANT) STANDARDS 

In addition to mastery standards, Section 10-14 m-r of Connecticut General 
Statutes requires that the Connecticut State Board of Education establish 
statewide standards for remedial assistance in order to meet two 
responsibilities: 

0 to identify and monitor the progress of students in need of remedial 
assistance in language arts/reading and mathematics as part of the 
EERA field assessments; and 

o to distribute EERA funds based on the number of needy students 

statewide, as well as for use in the Chapter 2 and Priority School 
District Grants. 

Students who score below the remedial standard(s) are eligible for services 
provided for in EERA legislation. Remedial standards were established by the 
State Board of Education acting on the recommendations of committees that 
represented Connecticut citizens and educators- The standard-setting 
committees recommended the following remedial standards: 

1. In mathematics, a student who answers fewer than 79 of the 144 items 
(55%) correctly Is required to receive further diagnosis by the local 
school district and. If necessary, to be provided with remedial 
assistance. 

2. In reading, a student whose Degrees of Reading Power (DRP) unit score 
is lower than 50 is required to receive further diagnosis and, if 
necessary, to be provided with remedial assistance. 

3. In writing, a student receiving a total holistic score less than 4 is 
required to receive further diagnosis by the local school district 
and, if necessary, to be provided with remedial assistance. 

The mastery and remedial standards were established by the State Board of 
Education on June 4, 1986. For a detailed explanation of the remedial 
standard-setting process, see Appendix D (p. 43). 



STATEHIDE ACHIEVEMENT GOALS 

In addition to mastery and remedial standards, statewide achievement goals 
have been established in the content areas of mathematics, reading (DRP) and 
writing. These goals represent high expectations and high levels of 
achievement for Connecticut public school students. 

The achievement goals are as follows: 

0 In mathematics, all students must master 31 of 36 objectives tested. 

0 In reading, a student must score a Degrees of Reading Power (DRP) 
unit score of 56 with 75% comprehension. 

0 In writing, a student must score a total holistic score of 7 on a 
scale of 2 to 8. 



STUDENT GRONTH OVER TIME 



The Connecticut Mastery Test (CMT) Program is designed to provide 
criterion-referenced information about the level of student mastery of 
objectives in grades four, six and eight* However, the basic scores reported 
for the mastery tests do not provide a system for evaluating achievement 
growth from grade four to grade six to grade eight ♦ This is so because 
mastery decisions are based on student performance (mastery/non-mastery) on 
objectives that are unique to grade level ♦ Mastery of objectives cannot be 
compared directly across grade levels and tests because of the differences in 
the number of objectives, curriculum content and levels of difficulty. In 
order to make valid interpretations across grade levels, the mastery test 
performance must first be linked using a procedure called vertical equating. 

Purpose of Vertical Equating 

Vertical equating is a psychometric technique for comparing tests at all 
ability levels. This is accomplished by putting them on a new scale which is 
common to the tests. Vertical equating is based on two assumptions. The 
first is that learning is continuous. The second is that instruction in each 
area is related to increased achievement in thctt area. These assumptions 
enable test developers to create a score scale that covers a wide range of 
content over several grades. The development of these "growth scales" is a 
common practice and has been used successfully in the development of a variety 
of achievement test batteries. The purpose of vertical equating is to provide 
one scale score system which can be used to compare performance across 
multiple grade levels. This score system enables test users to interpret test 
score information over time without altering the basic nature of the testing 
program. This achievement growth can be monitored over time on the basis of 
student performance on the CMT across grades. 



Development of Vertical Scales 

In order to develop a vertical scale, performance on the grade four, grade six 
and grade eight mastery tests was statistically linked. This was accomplished 
during the 1987 administration of the CMT using representative statewide 
samples of approximately 5,000 sixth-grade students and approximately 7,000 
eighth-grade students. Each group of students at grade six and grade eight 
was administered the appropriate on-grade level test form of the CMT along 
with one below-grade level section of the CMT. Specifically, each group of 
eighth-grade students took the grade eight test as usual and a part of the 
grade six test. Likewise, each sixth-grade group took the grade six test as 
usual along with a section of the grade four test. Each sample of students 
took only one below-level section of the CMT involving approximately one hour 
of additional testing time. Performance on the below-level items was not 
counted toward the CMT scores of individual students. For each of these 
linking samples, item difficulty estimates were obtained for the on-grade and 
below-grade level items by analyzing all items together as one test. Once 
items from the on-grade and below-grade level tests were linked, item 
difficulties from each level of the CMT were adjusted to a common metric to 
produce the vertical scale. 
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Vertical scales were established in the content areas of mathematics and the 
reading comprehension section of the language arts test* For each grade and 
content area, every correct score corresponds to a specific value on a common 
score scale (vertical scale)- Each of the vertical scales was constructed so 
that each scale score point represents the same theoretical achievement level 
whether derived from a score on the grade four test, a score on the grade six 
test or a score on the grade eight test- This allows valid interpretations of 
growth across time using tests differing in content, length and item 
difficulty- All items on the mathematics and reading comprehension tests were 
used in the development of the vertical scales- The writing and language arts 
tests were not scaled because of the nature of these assessment processes - 
The Degrees of Reading Power (DRP) test employs DRP unit scores which are 
already on a common scale across grades, obviating the need for any other 
development- (For more information see Congero, W-J-, 1989, The Development 
of Vertical Scales to Enhance the Evaluation of Assessment Data- Paper 
presented at the annual conference of the National Council of Measurement in 
Education, San Francisco, CA- This paper is available through the Student 
Assessment and Testing Unit of the Bureau of Evaluation and Student 
Assessment-) 

Scaled scores can be used to measure growth over time because CMT scores from 
all three grade levels have been placed on a common scale- These scales 
provide a means of monitoring students' academic progress from grade to 
grade- Before the scales were developed, it was difficult to assess the 
performance of groups of test takers as they moved from grade to grade because 
of differences in test length, curriculum content covered and levels of 
difficulty on the fourth-, sixth- and eighth-grade tests- 
Si nee students who took the fourth-grade test in 1988 subsequently took the 
sixth-grade test in 1990, change in test performance can be assessed across 
two years' time- Similarly, change in performance can be assessed for 1991 
sixth graders who took the grade four test in 1989- A summary of the overall 
growth in performance for these two groups of students in the content areas of 
mathematics and reading comprehension can be found on pages 30 and 31, 
respectively, of this manual- Students who took the fourth-grade test in 1986 
subsequently took the sixth-grade test in 1988 and the eighth-grade test in 

1990- Similarly, students who took the fourth-grade test in 1987 subsequently 
took the sixth-grade test in 1989 and the eighth-grade test in 1991- A 
summary of the overall growth in performance for these groups of students in 
the content areas of mathematics and reading comprehension can be found in the 

1991- 92 Grade 8 Summary and Interpretations Manual - 



NORMATIVE INFORMATION 

The CMT Program is designed to provide detailed information about fourth-, 
sixth- and eighth-grade students' mastery of specific skills and objectives- 
The provision of national norms with CMT results is intended to enhance the 
usefulness and flexibility of mastery test information by offering a bridge to 
conventional norm-referenced testing programs- The decision to provide 
normative information with the CMT does not change the essential purposes of 
our criterion-referenced testing program- The CMT will continue to be used 
for diagnostic and other instructional purposes with results reported at the 
student, classroom, school, district and state levels- 
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In particular, national norms provide greater: 

0 Test Economy, By providing national norms with CMT results, school 
districts can eliminate their standardized testing programs at these 
grades, thus saving money and undue testing time while retaining 
normative data* 

0 Test Efficiency. Federal compensatory programs require the 

systematic testing of students using instruments that can provide 
normative information. Because norms are provided with the CMT, 
school districts will not have to "double test" compensatory program 
students. This service allows for increased instructional time for 
these students. 

0 Test Interpretability. Criterion-referenced test (CRT) programs 

may be criticized because the public has difficulty Interpreting CRT 
performance. National norms will assist in the interpretation of CMT 
performance by providing a traditional benchmark with which the 
public is familiar. 



Development of Norms 

In order to provide estimated national norm-referenced data based on CMT 
performance, items on the CMT were statistically linked to items on a 
nationally norm-referenced test (NRT). Content-appropriate items from a 
nationally normed host test were included on the CMT to provide a common 
referent to both tests. Test equating procedures were then used to link CMT 
items with the normed test by placing all the items on a common scale. With 
this linkage in place, estimates of how the performance of Connecticut 
students compares to a national sample could be made. The NRT used to 
accomplish this task was the sixth edition of the Metropolitan Achievement 
Test (MAT-6), normed in 1986. The equating of the CMT to the MAT-6 enabled 
group summary scores on the CMT to be interpreted relative to the MAT-6 
nationally representative normative data. 

The CMT was initially equated to the MAT-6 during the pilot testing phase to 
investigate the relationship of the test content match between the two tests 
and the differential nature of the items included on the CMT and MAT-6. In 
addition, these preliminary data provided a benchmark by which the stability 
of the link could be monitored over time. The stability issue is monitored 
each year by readministering MAT-6 items during CMT administrations using 
representative statewide samples. The comparison of these data with prior 
information provides the information necessary to identify the instructional 
effects on student performance over time and to update the CMT/MAT-6 link as 
appropriate. This monitoring and updating ensures the continued accuracy of 
the normative estimates. 
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RESEARCH OPTIONS PROGRAM 



The Research Options Program is a free service provided by the Connecticut 
State Department of Education (CSDE) to help educators and educational 
policymakers gain access to the extensive information available from the 
Connecticut Mastery Test (CMT)* Participation in the Research Options Program 
is completely voluntary. 

The Research Options Program allows educators and educational policymakers 
(Ke.» superintendents, principals, researchers, evaluators and school board 
members) to benefit from customized research investigations designed to suit 
their individual needs or questions. Many school districts have taken 
advantage of the Research Options Program in previous years to successfully 
address special local concerns. 

The Research Options Program provides a number of ways of examining student 
achievement, as measured by the CMT, For example, one method is to compare 
aggregated student test scores obtained from the CMT in two or more categories 
of interest. Categories might include males and females, special program 
students compared to non-special program students, or any other comparison. 
These reports include tables that show the proportion of students mastering 
each objective, average number of objectives mastered and the achievement 
Indicators for students on each component of the test under consideration. 
These breakdowns allow district personnel to directly compare the performance 
of specific groups of students. In addition, graphics are provided, as 
appropriate, with each report in order to simplify the task of interpreting 
data. 

The Research Options component of the CMT has grown a great deal since the 
first study was performed on the Connecticut Basic Skills Proficiency Test 
almost a decade ago. This year, test directors and evaluators in 26 districts 
took advantage of this valuable resource to address questions of local 
interest. In addition, statewide programs such as Bilingual Evaluation, 
Chapter I and School Effectiveness have used the research options to obtain 
useful information for participants in over 100 districts. [For more 
information see Mooney, R.F., 1989, The Connecticut Mastery Test Research 
Options Program: The Application of State Criterion-Referenced Test Reports 
for Local Research Needs. Paper presented at the annual conference of the 
National Council of Measurement in Education, San Francisco, CA. See also the 
Research Options Handbook (1988) provided by the Connecticut State Department 
of Education. (These references are available through the Student Assessment 
and Testing Unit of the Bureau of Evaluation and Student Assessment.)] 



TEST ADMINISTRATION AND SCORING 

The regular administration of the Connecticut Mastery Test (CMT) for 1991 was 
conducted using Form D during a three-week period commencing on September 23, 
1991. Test sessions were conducted by local school district staff under the 
supervision of local test coordinators who had been trained by staff of the 
Connecticut State Department of Education (CSDE) and The Psychological 
Corporation (TPC). A student who took all subtests participated in 
approximately eight hours of testing. 
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The Grade 6 Connecticut Mastery Test had eight testing sessions* 



Mathematics I (60 minutes) 

Mathematics II (60 minutes) 

Mathematics III (60 minutes) 

Writing Sample (45 minutes) 

Degrees of Reading Power (70 minutes) 

Reading Comprehension (60 minutes) 

Listening Comprehension (45 minutes) 

Writing Mechanics/Study Skills (60 minutes) 

At the conclusion of the make-up testing period, answer booklets were returned 
to TPC in San Antonio, Texas for optical scanning and scoring, and then 
organized in preparation for holistic scoring workshops. 



Scoring of the Language Arts and Mathematics Tests 

The mathematics and language arts multiple-choice tests were machine-scored by 
TPC* Mathematics scores were reported for the total test as well as for 
mastery by each objective. Language arts scores were reported for mastery of 
each objective only. 



Scoring of the Writing Sample 

Every writing sample was scored by Connecticut educators using a technique 
known as the holistic scoring method. Holistic scoring is an impressionistic 
and quick scoring process that rates written products on the basis of their 
overall quality* It relies upon the scorers' trained understanding of the 
general features that determine distinct levels of achievement on a scale 
appropriate to the group of writing pieces being evaluated. All participants 
received on-site training and were required to demonstrate a clear 
understanding of the scoring criteria prior to actually scoring student 
essays. Each paper receives a final score between 2 and 8, where 2 represents 
a poor paper and 8 represents a superior paper. A thorough description of the 
training and scoring process, including sample papers representing different 
holistic scores, is presented in Appendix E (p. 49). 

In past years, the Connecticut Mastery Test writing sample was scored 
exclusively by Connecticut teachers. Following this year's scoring sessions, 
it was discovered that some of the Connecticut papers had not been scored 
accurately according to the established holistic scoring criteria. This 
problem was identified by department staff while conducting routine 
post-scoring checks. Further examination of the scoring process by department 
staff and contractors established that a portion of the papers in grades 6 and 
8 would need to be rescored. Accordingly, all grade 6 and grade 8 papers were 
submitted to a validation rescoring by our scoring contractor. This effort 
succeeded in confirming the accuracy of the initial holistic scores for the 
large majority of papers. Where a discrepancy was detected, the original 
scores were corrected to be made consistent with the established criteria. 
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Analytic Scoring 



All papers receiving holistic scores at or below the remedial standard of 4 
also received analytic scoring in four categories (traits): focus, 
organization^ support/elaboration and conventions* Analytic scoring is a 
thorough, trait-by-trait analysis of those components of a writing sample that 
are considered important to any piece of writing in any context* This scoring 
procedure can provide a comprehensive picture of a student's writing 
performance if enough traits are analyzed* It can identify those traits that 
make a piece of writing effective or ineffective* However, the traits need to 
be explicit and well defined so that the raters understand and agree upon the 
basis for making judgments about the writing sample* The analytic rating 
guide and sample marker papers for the analytic scoring are presented in 
Appendix F (p* 61)* 



Scoring of the Degrees of Reading Power (DRP) Test 

The DRP multiple-choice test was machine-scored by TPC* The scores reported 
are in DRP units* These scores identify the difficulty or readability level 
of prose that a student can comprehend* This makes it possible to match the 
difficulty of written materials with student ability* These scores can be 
better interpreted by referring to the readability levels of some general 
reading materials as shown below: 

0 Elementary textbooks (grades 5-7) 45-65 DRP Units 

0 Personality Section - teen magazines - 55 DRP Units 

0 Adolescent fiction - 55 DRP Units 

A much more extensive list of reading materials is contained and rated in the 
Readability Report , Seventh Edition, published by The College Board* 

The conversion between DRP unit scores and raw scores can be made from the 
tabled values obtainable through the Student Assessment and Testing Unit of 
the Bureau of Evaluation and Student Assessment* 



SCH(X)L DISTRia TEST RESULTS REPORTING 

The CMT school district reports are designed to provide useful and 
comprehensive test achievement information about districts, schools and 
students* Four standard test reports are generated to assist superintendents, 
principals, teachers, parents and students to understand and use 
criterion-referenced test results* Appendix G (p* 67) presents samples of the 
district, school, class and parent/student diagnostic score reports* 



FALL 1991 STATENIDE TEST RESULTS 

The Grade 6 Connecticut Mastery Test provides a comprehensive evaluation of 
student performance on specific skills that Connecticut educators feel are 
important at the beginning of sixth grade* The mastery test's greatest 
instructional utility lies in its identification of areas of student weakness 
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and strength. This report profiles the statewide results. Each school 
district also receives a full complement of reports that identify patterns of 
academic strength and weakness at the district, school, classroom and 
individual student levels. 

Chart 1 (p. 12) gives a statewide summary of the average number of objectives 
mastered (mathematics and language arts), average writing and reading scores, 
the number of students scored, the number of students scoring at or above the 
remedial standard and goal (where applicable) and the percent of students 
scoring at or above the remedial standard and goal (where applicable). 



The following are highlights of the 1991 Grade 5 CMT results: 
MATHEMATICS 

0 Sixth graders mastered an average of 24.7 of 36 objectives tested, up 
slightly from last year's figure of 24.6. 

0 A total of 84.211 of the students scored at or above the remedial 
standard, down slightly from last year's figure of 84.5%. 

0 A total of 31.7% of the students scored at or above the mathematics 
goal, an increase from last year's figure of 30.0%. 

LANGUAGE ARTS 

0 Sixth graders mastered an average of 8.1 of 11 objectives tested, 
representing no change from last year. 

WRITING 

0 Sixth graders averaged 4.8 on a scale of 2 to 8, up slightly from 
last year's 4.6. 

0 A total of 82.2% of the students scored at or above the remedial 
standard, an increase from last year's figure of 79.7%. 

0 A total of 12.4% of the students scored at or above the writing 
goal, an increase from last year's figure of 10%. 



READING 

0 Sixth graders averaged 57 units on the Degrees of Reading Power (DRP) 
test, representing no change from last year. 

0 A total of 75.7% of the students scored at or above the remedial 
standard, down slightly from 76.4% last year. 

0 A total of 60.1% of the students scored at or above the reading goal, 
representing a slight increase from last year. 
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CHART 1 

1991 CONNECTICUT MASTERY TEST RESULTS 
GRADE 6 STATEWIDE SUMMARY 



AVERAGE 

NUMBER OF NUMBER OF STUDENTS AT OR ABOVE STUDENTS AT OR ABOVE 

OBJECTIVES STUDENTS REMEDIAL STANDARD* STATE GOAL** 

SUBJECT MASTERED SCORED NUMBER PERCENT NUMBER PERCENT 

MATHEMATICS 24.7 33,343 28,088 84.2% 10,556 31.7% 

LANGUAGE ARTS 8.1 32,940 



AVERAGE 
HOLISTIC SCORE 

WRITING SAMPLE 4.8 33,138 27,234 82.2% 4,100 12.4% 



AVERAGE DRP 
UNIT SCORE 

READING 57 33,353 25,257 75.7% 20,029 60.1% 



* MATHEMATICS REMEDIAL STANDARD = 79 ITEMS CORRECT 
WRITING REMEDIAL STANDARD = 4 
READING REMEDIAL STANDARD = 50 DRP UNITS 



** MATHEMATICS GOAL = 31 OBJECTIVES MASTERED 
WRITING GOAL = 7 
READING GOAL = 56 DRP UNITS 
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Mathematics 



In mathematics, sixth graders mastered an average of 24*7 objectives, or 
68.6%. of the 36 objectives tested. While the state's goal is that all 
students master every objective, an interim standard (31 of 36 objectives 
mastered) has been established which represents a high level of mathematics 
achievement. Chart 2 (p. 15) illustrates that, statewide, students 
demonstrated strength (85% or more students achieving mastery) in the 
objectives of ordering whole numbers; identifying place value and using 
expanded notation; extending patterns with numbers or attributes; computation 
with whole numbers and money amounts; multiplication/division facts; 
interpreting graphs, tables and charts; and identifying graphs best fitting 
given data. However, students did not perform as effectively (fewer than 50% 
of the students achieving mastery) on objectives of renaming whole numbers by 
regrouping; converting between mixed numbers and improper fractions; 
estimating sums/differences of fractions and mixed numbers; solving problems 
with extraneous information; estimating lengths and areas; and determining 
areas and perimeters. 

There continues to be a consistent pattern throughout the mathematics subtests 
of student strengths in primarily computational skills and easy one-step 
applications. These strengths are offset by an equally clear pattern of 
student weaknesses on higher order objectives involving more than routine 
conceptual understanding or simple application of skill. For example, 
students are consistently strong in their ability to recall number facts and 
compute with whole numbers. However, there is consistent weakness in working 
with fractions, making estimates and solving problems with extraneous 
information. 

Students getting fewer than 79 questions correct on this 144-question 
mathematics test (15.8% of the students tested) were identified as needing 
further diagnosis and possible remedial instruction. 



Language Arts 

In language arts, sixth-grade students averaged 8.1 objectives, or 73.6% of 
the 11 objectives tested. The state's goal is that all students master every 
objective. Chart 3 (p. 16) illustrates that students did reasonably well on 
study skills and writing mechanics, except for capitalization and 
punctuation. However, weaknesses were found in reading comprehension and to 
some extent in listening comprehension. These results indicate that students 
need to learn more effective comprehension strategies while simultaneously 
being exposed to a wide variety of reading selections. 
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In writing, sixth-grade students averaged 4*8 points on a scale of 2 through 
8. The state's goal is that all students be able to produce an organized, 
well-supported piece of writing, that is. a holistic score of 7 or S. Chart 4 
(p. 17) illustrates that 13X of the students produced an organized, 
well-supported piece of writing (scores of 7 or 8). and an additional 4U 
produced a paper which is generally well-organized (scores of 5 or 6). A 
total of 29X of the students scored a 4. which indicates minimally proficient 
writing, while the remaining 18% scored below the remedial standard (scores of 
2 or 3)* 

In reading (Degrees of Reading Power test), sixth-grade students averaged 57 
units on a scale of 15 through 99. The state's goal is that all students be 
able to read with high comprehension those materials typically used at the 
sixth grade or above; that is. at least 56 on the DRP unit scale. Chart 5 
(p. 18) illustrates that 60% of the students scored at least 56 on the DRP 
score scale. 16% scored between 50 and 55 and 24% scored below the remedial 
standard of 50. The average score of 57 suggests that Connecticut sixth 
graders typically can read and comprehend materials normally used up to grade 
six. To further improve reading performance, continued emphasis needs to be 
placed on reading nonfiction materials during the primary and intermediate 
grades. 
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CHART 4 
WRITING SAMPLE: 
PERCENT OF STUDENTS AT EACH SCORE POINT 
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This bar chart illustrates the distribution of students who received each holistic 
writing score, statewide. Holistic writing scores are interpreted as follows: a student 
who scores 7 or 8 has produced a paper which is well written with developed 
supportive detail; a student who scores 5 or 6 has produced a paper which is 
generally well organized with supportive detail; a student who scores 4 is minimally 
proficient; and a student who scores 2 or 3 is in need of further diagnosis and 
possible remedial assistance. 
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CHART 5 

DEGREES OF READING POWER® (DRP)®: 
PERCENT OF STUDENTS AT SELECTED RANGES OF 

DRP UNIT SCORES 
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This bar chart illustrates the distribution of students, statewide, scoring in each of three 
Degrees of Reading Power (DRP) score categories. DRP score categories are interpreted 
as follows: a student who scores 56 DRP units or above has met the statewide Reading 
Goal and can read, with high comprehension, materials which are typically used at grade 6 
or above; a student who scores 50-55 DRP units can read, with high comprehension, 
materials which are typically used below grade 6 but above the Remedial Standard; and a 
student who scores 49 DRP units or below is in need of further diagnosis and possible 
remedial assistance. 
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COMPARISON OF 1986 THROUGH 1991 TEST RESULTS 



Charts 6-12 (pp. 21-27) address the comparison of the 1986 "hrough 1991 test 
results. Charts 6 (p. 21), 9 (p. 24) and 10 (p. 25) present a comparison of 
statewide average scores on the four subtests, a comparison of the percent of 
students scoring at or above the remedial standard, and a comparison of the 
percent of students scoring at or above the statewide goals, respectively. 
The remaining four charts provide a comparison of the percent of students 
achieving mastery in each mathematics objective (Chart 7, p. 22) and each 
language arts objective (Chart 8, p. 23), a comparison of student achievement 
in relation to the remedial standards (Chart 11, p. 26), and a comparison of 
student achievement in relation to the statewide goals (Chart 12, p. 27). 

Chart 6 (p. 21) shows that the statewide average scores remained steady or 
Increased slightly in all areas tested, when 1991 results are compared to 1986 
results. In mathematics, the average number of objectives mastered increased 
from 23.1 in the initial 1986 assessment to 24.7 in 1991. Mathematics scores 
have increased slightly in each of the test administrations indicating a 
steady, albeit small, positive trend. DRP reading performance has also moved 
in a positive direction. While the average DRP score was unchanged from 1988 
to 1991, there has been a one point increase in other years moving from 55 in 
1986 to 57 in 1988. In language arts, the average number of objectives 
mastered has increased from 7.5 objectives mastered in 1986 to 8.1 mastered in 
1991. Student performance on the writing samples showed some progress from 
1986 to 1991, with the average holistic score increasing from 4.7 to 4.8. 

Chart 7 (p. 22) lists the percent of students at mastery for each of the 36 
mathematics objectives. Of the 34 objectives assessed from 1986 to 1990, 22 
objectives have shown a gain in percent of students at or above mastery, 7 
have declined and 5 are unchanged. Note that 2 objectives were changed in the 
1990 assessment (see pp. 1-2). A comparison of the 1991 and 1986 results 
shows large gains (at least 10 percentage points) in the percent of students 
meeting the mastery standard in the following objectives: renaming whole 
numbers by regrouping, rounding whole numbers less than 100,000, multiplying 
and dividing multiples of 10 and 100 by 10 and 100, finding fractional parts 
of whole numbers, identifying number sentences from problems, estimating a 
reasonable answer to a given problem, solving 1-step problems with fractions, 
measuring/determining perimeters and areas and estimating lengths and areas. 

Chart 8 (p. 23) lists the percent of students at mastery for each of the 11 
language arts objectives. From 1986 to 1991, 10 objectives have shown a gain 
in percent of students at or above mastery and one objective has shown a 
decline. When 1991 results are compared with 1986, the only area which showed 
a substantial decline was capitalization and punctuation, which dropped 11 
percentage points. 



Chart 9 (p» 24) compares the percent of students who scored at or above the 
remedial standard in mathematics, writing and reading (DRP) for 1986 through 
1991. In each content area there has been a gain in the percent of students 
meeting the remedial standard over the six CMT administrations indicating 
continued movement in a positive direction. The remedial standard for 
mathematics is 79 out of 144 items correct, A 3 percentage point increase in 
performance at or above the remedial standard from 1986 (8U) to 1991 (84%) 
was observed* The remedial standard for writing is 4 on a scale from 2 to 8. 
A 4 percentage point increase in writing performance at or above the remedial 
standard was reported from 1986 (78%) to 1991 (82%), The remedial standard 
for reading (DRP) is 50 DRP units with 751 comprehension. A 7 percentage 
point increase in performance at or above the remedial standard was reported 
from 1986 (69%) to 1991 (76%) • 

Chart 10 (p. 25) compares the percent of students scoring at or above the 
statewide goals in mathematics, writing and reading from 1986 through 1991. 
In mathematics, the goal is 31 of 36 objectives mastered. There was a 9 
percentage point increase in performance at or above the statewide goal from 
1986 (23%) to 1991 (32%). In writing, the goal is 7 on a scale of 2 to 8. 
The percent of students scoring at or above the statewide standard decreased 
from 15% in 1986 to 12% in 1991. In reading (DRP) the statewide goal is 56 
DRP units with 75% comprehension. There was a 7 percentage point increase in 
performance at or above the goal from 1986 (53%) to 1991 (60%). 

Chart 11 (p. 26) is a comparison of student achievement in relation to the 
remedial standards from 1986 through 1991. Over the six-year period, the 
percent of students at or above the remedial standard on all three tests 
(mathematics, reading, writing) has increased from 56.6% in 1986 to 62.9% in 
1991, while the percent of students below the remedial standard on all three 
tests has declined from 7.2% in 1986 to 5.0% in 1991. The percent of students 
below the remedial standard on one or more subtests has also dropped from 
42.7% in 1986 to 35.7% in 1991. 

Chart 12 (p. 27) is a comparison of student achievement in relation to the 
goals for 1986 through 1991. Over the six-year period, there has been a 
slight decrease in the percent of students reaching the statewide goal on all 
three tests (mathematics, reading, and writing). The percent of students 
below the statewide goal on all three tests has declined from 42.4% in 1986 to 
34.7% in 1991. The percent of students above the statewide goal on one or 
more subtests has increased from 56.5% in 1986 to 63.0% in 1991. 



Test Results by District 

Appendices H and I address the comparison of test scores by school district. 
Appendix H (p. 81) and Appendix I (p. 89) present a listing of the mathematics 
and language arts test results, respectively, for each Connecticut school 
district. In each appendix, school districts are listed alphabetically, 
followed by regional school districts. The Type of Community (TOC) 
designation in the second column and the Education Reference Group (ERG) 
designation in the third column indicate the TCX; and ERG groups with which 
each district or school has been classified. Definitions of the TOC and ERG 
classifications are provided in Appendix J (p. 97) and Appendix K (p. 99), 
respectively. TOC and ERG summaries follow the alphabetical listings of 
school district results in mathematics and language arts. 




-20- 



X 

o 
o 

X 

CO 
00 

o> 

oc 

O 

LL 

(0 
lU 
flC 
O 
O 

LU 
Q 

i 

LL 
O 

z 

o 

(0 

£ 

o 
o 



00 



00 



ay 
1^ 



o 

00 



00 



CD 



C\J 



a3H3isvi^ SBAiioarao 



CM 



CD 



00 
CO 
CM 



CO 
CVJ 



CO 
CO 



O 
CO 



in 



o 



in 



in 



a3H3isviN SBAiioarao 



o 

^ CO 

cc 

00 

O) LU 

< 



00 



CO 
00 



o 



CO 
O 
ay H 

LU 
00 X 
00 lZ 



00 



CO 
00 

ay 



in 



1^ 
in 



CD 

to 



in! 



CJ5 



O 
CD 



O 

in 



o 



o 

CO 



aaoos iiNfi daa 3ovh3av 



ooi 

'1 



CO 



00; 



ay\ 



o in 

CVJ T- 



1 



] 



ooh-coin'^cocvjT-o 



aaoos oiisnoH aovaaAV 



ay 
ay 



o 



ay 



ay 
ay 



o> 
ay 



ay 

00 



Q 

00 < 

2 OC 



00 

ay 



CO 
00 

ay 



00 5 
00 > 

ay > 



00 

ay 



CD 
00 



-21- 



CHART? 

MATHEMATICS: COMPARISON OF THE PERCENT OF STUDENTS 
ACHIEVING MASTERY IN EACH OBJECTIVE FOR 1986 THROUGH 1991 



OBJECTIVE 


PERCENT OF STUDENTS 
AT MASTERY 


PERCENTAQE 
POINT 
GAIN FROM 
1986 TO 1991 




1986 


1987 


1988 


1989 


1990 


1991 




CONCEPTUAL UNDERSTANDINGS 
















1. ORDER WHOLE NUMBERS LESS THAN 100.000 


94% 


96% 


96% 


96% 


94% 


94% 


0% 


2. IDENTIFY DIGIT VALUE/USE EXPANDED NOTATION 


90% 


92% 


92% 


92% 


90% 


90% 


0% 


3. RENAME WHOLE NUMBERS BY REGROUPING 


17% 


27% 


28% 


42% 


39% 


40% 


23% 


4. ROUND WHOLE NUMBERS LESS THAN 100.000 


55% 


64% 


65% 


62% 


69% 


69% 


14% 


5. MULTIPLY/DIVIDE MULTIPLES OF 10/100 BY 10/100 


67% 


70% 


70% 


72% 


79% 


78% 


11% 


6. ID EQUIV FRACTIONS AND MIXED #'S USING PICTURES 


59% 


54% 


54% 


59% 


57% 


57% 


-2% 


7. IDENTIFY EQUIVALENT FRACTIONS AND MIXED NUMBERS 


51% 


50% 


49% 


48% 


59% 


58% 


7% 


8. CONVERT BETWEEN MIXED #"S & IMPROPER FRACTIONS 










42% 


44% 




9. IDENTIFY DECIMALS (.01 TO 2.99) FROM PICTURES 


53% 


54% 


56% 


70% 


54% 


56% 


3% 


10. EXTEND PATTERNS INVOLVING NUMBERS/ATTRIBUTES 


91% 


92% 


92% 


94% 


93% 


93% 


2% 


11. ID APPROP PROCEDURE TO ESTIMATE WHOLE # COMP 


82% 


81% 


82% 


78% 


76% 


76^o 


-6% 


COMPUTATIONAL SKILLS 
















12. ADD/SUBT NUMBERS <1 00.000 & AMOUNTS < $100 


87% 


90% 


89% 


89% 


89% 


88% 


1% 


13. KNOW MULTIPLICATION AND DIVISION FACTS 


95% 


93% 


92% 


97% 


96% 


96% 


1% 


14. MULTIPLY WHOLE NUMBERS AND MONEY AMOUNTS 


93% 


90% 


89% 


90% 


92% 


92% 


-1% 


1 5. DIVIDE 2- AND 3-DlGIT NUMBERS BY 1 -DIGIT NUMBERS 


81% 


78% 


78% 


77% 


78% 


77% 


-4% 


16- ADD/SUB FRACTIONS - LIKE DENOMINATIONS 


81% 


66% 


66% 


76% 


81% 


81% 


0% 


ADD FRACTIONS WITH LIKE DENOMS. WITH REGROUPING 


52% 


60% 


60% 


63% 






" 


ADD/SUBTRACT FRACTIONS WITH UNLIKE DENOMINATORS 


38'>/o 


39% 


39% 


3(y>/o 








17. FIND FRACTIONAL PARTS OF WHOLE NUMBERS 


28% 


40% 


40% 


32% 


59% 


59% 


31% 


18. ESTIMATE SUMS/DIFFS OF WHOLE NUMBERS/S AMTS 


56% 


55% 


54% 


63% 


55% 


56% 


0% 


19. ESTIMATE PROD/OUOT OF WHOLE NUMBERS/$ AMTS 


61% 


59% 


59% 


57% 


52% 


53% 


-8% 


20. ESTIMATE SUMS/DiFFS OF FRACTS AND MIXED #'S 










30% 


31% 




PROBLEM SOLVING/APPLICATIONS 
















21. INTERPRET GRAPHS/TABLES/CHARTS 


89% 


88% 


88% 


86% 


87% 


88% 


-1% 


22. IDENTIFY GRAPH THAT BEST ILLUSTRATES DATA 


89% 


92% 


92% 


95% 


94% 


94% 


5% 


23. IDENTIFY NUMBER SENTENCES FROM PROBLEMS 


66% 


70% 


71% 


69% 


75% 


76% 


10% 


24. SOLVE 1 -STEP PROBS INVOLVING WHOLE NUMBERS & S 


79% 


79% 


80% 


78% 


83% 


83% 


4% 


25. SOLVE PROBLEMS INVOLVING MAKING CHANGE 


82% 


82% 


82«».o 


81% 


84% 


83% 


1% 


26. SOLVE 1 -STEP PROBLEMS INVOLVING FRACTIONS 


43% 


53% 


53% 


48% 


64% 


64% 


21% 


27. SOLVE 2-STEP PROBS INVOLVING WHOLE NUMBERS & S 


57% 


56% 


55% 


60% 


61% 


61% 


4% 


28. ESTIMATE REASONABLE ANSWER TO A GIVEN PROBLEM 


43% 


52% 


52% 


47% 


55% 


56ro 


13% 


29. IDENTIFY/SOLVE EXTRANEOUS INFO PROBLEMS 


38% 


39% 


39% 


41% 


45% 


46% 


8% 


30, IDENTIFY MISSING INFO IN PROBLEM SITUATIONS 


75% 


75% 


75% 


71% 


71% 


72% 


-3% 


31. SOLVE PROCESS PROBLEMS-DATA ORGANIZATION 


63% 


64% 


64% 


65% 


66% 


66% 


3% 


MEASUREMENT/GEOMETRY 
















32. IDENTIFY GEOMETRIC FIGURES 


72% 


80% 


80% 


83% 


72% 


72% 


0% 


33. MEASURE/DETERMINE PERIMETERS AND AREAS 


33% 


38% 


39% 


36% 


43% 


43% 


10% 


34. ESTIMATE LENGTHS AND AREAS 


33% 


38% 


37% 


45% 


48% 


49% 


16% 


35. PICK APPROP METRIC/CUSTOMARY UNIT AND MEASURE 


65% 


62% 


63% 


60% 


70% 


70% 


5% 


36. DETERMINE ELAPSED TIME 


52% 


53% 


55% 


54% 


52% 


53% 


1% 


* = Objective added in 1990 assessmeni 
'* = Objective dropped in 1990 assessment 
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CHART 8 

LANGUAGE ARTS: COMPARISON OF THE PERCENT OF STUDENTS 
ACHIEVING MASTERY IN EACH OBJECTIVE FOR 1986 THROUGH 1991 



OBJECTIVE 


PERCENT OF STUDENTS 
AT MASTERY 


PERCENTAGE 
POINT 
GAIN rriUM 
1986 TO 1991 




1986 


1987 


1988 


1989 


1990 


1991 




WRITING MECHANICS 
















1 . CAPITALIZATION AND PUNCTUATION 


71% 


75% 


74% 


68% 


60% 


60% 


-11% 


2. SPELLING/HOMONYMS/ABBREVIATIONS 


73% 


73% 


72% 


76% 


83% 


82% 


9% 


3. AGREEMENT 


78% 


77% 


77% 


82% 


80% 


80% 


2% 


4. TONE 


76% 


82% 


827o 


77% 


85% 


85% 


9% 


STUDY SKILLS 
















5. LOCATING INFORMATION 


83% 


79% 


73% 


78% 


84% 


84% 


1% 


6. NOTETAKING AND OUTLINING 


73% 


75% 


74% 


59% 


81% 


81% 


8% 


LISTENING COMPREHENSION 
















7. LITERAL 


65% 


67% 


66% 


65% 


73% 


72% 


7% 


8. INFERENTIAUEVALUATIVE 


65% 


82% 


82% 


70% 


73% 


73% 


8% 


READING COMPREHENSION 
















9. LITERAL 


54% 


65% 


65% 


56% 


66% 


65% 


11% 


10. INFERENTIAL 


55% 


57% 


57% 


55% 


61% 


61% 


6% 


11. EVALUATIVE 


56% 


62% 


63% 


55% 


62% 


63% 


7% 
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CHART 9 

COMPARISON OF THE PERCENT OF STUDENTS 
SCORING AT OR ABOVE THE REMEDIAL STANDARD 
IN EACH SUBJECT AREA FOR 1986 THROUGH 1991 
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CHART 10 

COMPARISON OF THE PERCENT OF STUDENTS 
SCORING AT OR ABOVE THE GOAL 
IN EACH SUBJECT AREA FOR 1986 THROUGH 1991 
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The State Department of Education advises against comparing scores between and 
among school districts. It is more meaningful to compare district results 
longitudinally within each district. It is also not appropriate or meaningful 
to sum across the different tests and subtests for comparative purposes 
because of differences in test length, mastery criteria and remedial 
standards. These comparisons are inappropriate because it is impossible to 
identify, solely on the basis of this information, how the average student has 
performed in the districts being compared. Average scores and standard 
deviations provide more appropriate comparative information on how well the 
average student is performing, although many factors may affect the 
comparability of these statistics as well. 



Normative Results 

Normative information is provided to indicate how well the average student in 
Connecticut performs compared to a national sample of students. Norms have 
been available for the mathematics test, the language arts test and the 
reading comprehension test since 1987< This year, for the second year, 
normative information is also being provided for mathematics problem solving. 
These norms are based on links established between the CMT and the sixth 
edition of the Metropolitan Achievement Test (MAT-6). The norms are expressed 
in percentile ranks which provide estimates of group performance relative to 
the performance of the national MAT-6 norm group. Percentile ranks range from 
1 to 99. A percentile rank of 50 represents the score that divides the norm 
group into two equal parts; half scoring below and half scoring above this 
value. Each reported percentile rank represents the performance of a 
nationally representative sample of students in relation to Connecticut 
student performance. 

The following are the total estimated norms for the grade six statewide 
averages. In the content areas of mathematics, language arts and reading 
comprehension (not DRP), data are provided for the 1987 through 1991 
administrations. Normative information in the content area of mathematics 
problem solving is presented for the 1990 and 1991 administrations only. 

Grade Six 

mi 

Total Mathematics 66 
Language Arts 65 
Reading Comprehension 57 
Mathematics Problem Solving 



Patterns In the data are summarized below. 

0 In each content area and administration year, the mean national 

percentile rankings of Connecticut students substantially exceed the 
national average (50th percentile rank). 

0 The norms for language arts have remained similar to one another over 
the five years with percentile ranks ranging from 63 to 66 in value. 
Reading comprehension performance continues to be lower than either 
mathematics or language arts when compared to a national sample, with 
percentile ranks ranging from 56 to 58 from 1987 to 1991. 



65 
66 
58 



71 
65 
56 



129Q 
64 
65 
58 
63 



mi 

65 
63 
58 
62 



0 With the exception of mathematics increasing to 71 in 1989, the 

percentile ranks within each content area are quite stable across the 
five years, differing in value by no more than two points. 



It should be pointed out that these norms provide a way to interpret the 
performance of the average Connecticut student relative to a national sample. 
They do not address the issue of how Connecticut, as a state, compares to 
other states* The fact that, in 1991, the average Connecticut student is at 
the 65th percentile in mathematics does not mean that the state as a whole 
would be in the 65th percentile if it were compared to other states. A 
state-by-state achievement testing program has been endorsed by the Council of 
Chief State School Officers (CCSSO) and the National Governors* Association 
(NGA) and is in progress using the National Assessment of Educational Progress 
(NAEP) Program. Connecticut participated In the 1990 trial state assessment 
for mathematics at grade eight. Results of this assessment were released June 
6, 1991 at a national press conference in Washington, D.C. In addition, 
Connecticut participated in the 1992 trial state assessment in grades four and 
eight. 



Norms Available to Districts 

Total mathematics, language arts, reading comprehension and mathematics 
problem solving norms can also be provided for groups of students at the 
district level. Each year all districts are notified by the CMT contractor 
that norms for their own districts and/or schools within their districts are 
optionally available. In addition, districts are offered all materials and 
directions to hand-calculate norms for groups of students within their 
districts (e.g., Chapter I students). There is no charge for either of these 
services. Any district that requests this information receives it directly 
from the CMT contractor. No district receives normative information unless it 
is specifically requested by the superintendent. Approximately one half of 
Connecticut school districts have requested norms In the past. 



Longitudinal Results 

In order to interpret student performance across grade levels, vertical scales 
were developed in the areas of mathematics and reading comprehension. Scaled 
scores can be used to measure growth over time because CMT scores from all 
three grade levels have been placed on a common scale. These scales provide a 
means of monitoring students' academic progress from grade to grade. Before 
the scales were developed, it was difficult to assess the performance of 
groups of test takers as they moved from grade to grade because of differences 
In test length, curriculum content covered and levels of difficulty on the 
fourth-, sixth- and eighth-grade tests. 

Since students who took the fourth-grade test in 1988 subsequently took the 
sixth-grade test in 1990» change in performance on the test can be assessed 
across two years' time for the group. Similarly, change in performance can be 
assessed for 1991 sixth graders who took the grade four test In 1989. Chart 
13 (p. 30) and Chart 14 (p. 31) present overall growth in performance for these 
two groups of students in the content areas of mathematics and reading 
comprehension. These results show meaningful growth in both mathematics and 
reading comprehension for both groups of students from grade four to grade six. 



-29- 



ERLC 



<0 

Q 
< 

GC 
O 

O 
H 

T- UJ 

oc < 



X 

o 



S 

0) 

o 

UJ 
X 



ERIC 



CO 

o 
o 

fl. o 

S o «^ 



O 

t: 

CD 
CL 



05 (/) 

IS 

CO "O 05 
O < 

EO) c 

■co<D< 

CO j= o 
> c c 

^ ^ CO 

CO 00 c 
CD c: 

O (D 2> 



Q 



3 

il 



1^1 
S o « 



.111: 



0) ^ 
> CO 
<D CD 



(A 

E o 
o <d 

Q> 0 



w Q- 
c a> 
o <^ 
.y X 

Q) ? 

°^ 

0) c 

o c 

X 0) 

0) 5 

CJ)0 

iS§ 

P' 

3 (0 

Q.^ 
0) *- 
O Q- 
C Q) 
O O 
O X 
•D ^ 
CO « 
0:=: 

OQ CO 



0) 

« 

Sf 

c c 

CO "O 

• c 

CO 

? i5 
S c 

0) 



> CO 

0) 10 

-iCO 



I > 

CO 3 

II 

c o 

<0 CO 

~ o> 

CO c 

■<5 £ 
qj CO 

■Q to 
S'O 

CO c 
W => 
Q)'0 

c TO 

3^ 



o 

^ E 
^ o 

<D 



|5 

a, c 
o o. 

Q,CD 

Q> c 

J2 > 
5 o 



= 10 

— in 

> CO 

_j<o 



CD 

0?2 



I / 



E o 

CO <y 



o)E 

?i 
co-5 

CO 



Q. Ji? 



= CO 

£0 

3 O 

0) 0) 
OF 

O y} 

^E 

0) 

CO ^ 
CO O 
2 Q. 



0) CD 



CO 



2c 
5 <» 

°§ 

CO 

o TO 
Q> 
w E 

t| 

CO c 

£?, 

<D CO 
T3 D> 
C C 

C 
O TO 



I? 



m C 
<l^-i CO 



to 



CD 

•DO 

(0 o> 

o2 



^ TO 

00 o 
CM CO 



CO 3 



Tj- CD 



< 



CD 
O ^ 

o) = 05 
<o $ c 



CO « 



00 



CD 

to 

CD 



CD 
CO 
CD 



CD 
O 
CD 



O 
O 
10 



8J00S aieos 



-30- 



LU 
O 
< 

o 
o 

H 

LU 
Q 
< 

GC 

H Z 

50 

z 



o 



lU 
X 
LU 

DC 

a. 

o 
o 

o 

z 

o 
< 

UJ 



(0 

2 
o 

o 
CO 

5 ^ 

CO o 
~ 

CO 2i2 

iS^ 

o c c 

O 

^ ^ ~ 

OC CD< 

<D (DTT 
■D-O 05 

"i 2^2 
So^ 

<D C cC 

< c 
OF .2 

o< 
-ioo 

CO 02 



'c 
E 



■a 
c 
CO 

T- CO 



o 

o 



o 



^ o 
00 »- 

T- O) 

o 02 



c 



CO CO 



3 

CO 



0) 



c 

p 



^1 
as 



-9^ 



0.^ 



QJ 3 



CO o 



> in 



> Q. 

<rt — 

o .E 
23 

CO o 
o <o 

> o 

Si 

- c 



I o 



V) 

CO . 
0) CO 

•2 E 

o r; 
</) JO 

c 5 

CO CO 

£ 

_ w 
£§. 
^3 

— Q. 
CD . 

*- CO 
■DO) 

<» 

<D O 
I- 

oi2 
o c 

si 

a> o) 

CO .2. 




- a> 

to 

a)ca 

£ E 
c w 
CO o 

H 

lo g 



11 
11 



o o 

.•El = 0 



to 



CO ! 



g a> Q- 
coii CO 



— O) 

~ o 
a)CD 



c 

I 
•0 



^ c 



CD 

"OO 

(0 o> 

o2 



^ CO o 
CO (/) </) 



00 



8|i 

Q. 
< 



o) y CO 
CD 5 p 



s 



V> Q 

CO <o 
9JO0S 9|B0S 



CM 



O 
o 
to 



-31- 



Chart 13, for example, shows that the average statewide performance in 
mathematics, for the group of students who took the fourth-grade test in 1988 
and the sixth-grade test in 1990, has moved in a positive direction from Level 
of Understanding II to Level of Understanding III over the two-year period. 
While initial results are encouraging, it is still premature to draw 
definitive conclusions about how much growth to expect as students progress 
from grade to grade. Such conclusions are possible only after the program has 
been in effect for several years. It should be noted that each sixth-grade 
group differs, to some extent, from its respective fourth-grade group because 
some students entered, while other students exited the Connecticut public 
school system over the two-year period. 



Participation Rate Results 



Appendix L (p. 103) presents the number of sixth-grade students in each 
district and the percents of students who participated in the grade six 
mastery testing during the fall 1991 statewide administration. Appendix L 
also shows the percent of students exempted from CMT testing. The 
alphabetical listing of districts provides the following information for each 
district: 



Column 1 The name of the district 

Column 2 The total sixth-grade population at the start of mastery 
testing 

Column 3 The number of students eligible for testing 
Column 4 The percent of total population exempted from testing 
Columns 5-8 The percent of eligible students tested in each content 
area 



The results in Appendix L illustrate that participation rates by school 
district on the sixth-grade CMT were quite high, with only a few exceptions. 
However, the high percentage of students exempted from the CMT, statewide, 
combined with the large variation in exemption rates among districts, has 
raised concerns about the fair application of exemption procedures and its 
impact on students. The Department is currently examining the impact of the 
exclusion provisions on the CMT programs for Special Education and bilingual 
students. The results from these analyses are available from the Division of 
Research, Evaluation, and Assessment. 
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Test Construction 



The development of the sixth-grade criterion-referenced mastery test required 
the formation of seven statewide advisory committees. These included the 
Mathematics and Language Arts Advisory Committees, the Psychometrics Advisory 
Committee, the Bias Advisory Committee, the Connecticut Student Assessment 
Advisory Committee (formerly the Mastery Test Implementation Advisory 
Committee), and two standard-setting committees, one for mathematics and one 
for language arts. These committees were comprised of representatives from 
throughout the state. Members were selected for their area of expertise. 
Approximately 150 Connecticut educators participated on the mastery test 
committees which met over 80 times during the first 18 months of test 
development. (See Acknowledgements, p. v and p. 48.) 

Beginning in the spring of 1985, content committees in both language arts and 
mathematics participated in each stage of the test development process, 
including assisting the State Department of Education in the selection of The 
Psychological Corporation as its test contractor. First, the content 
committees reviewed the curriculum materials prevalent throughout the state 
and the scope of the national tests in use in Connecticut at the respective 
grade levels. Additional resources included the Connecticut curriculum guides 
in mathematics and language arts, developed in 1981, as well as the results of 
recent Connecticut Assessment of Educational Progress (CAEP) assessments in 
mathematics and language arts. Next, the committees identified sets of 
preliminary mathematics and language arts objectives which reflected existing 
curriculum materials and the goals of the mastery testing program. The 
content committees defined an objective as an operational ized learning outcome 
that was fairly narrow and clearly defined. 

Four criteria were used in identifying the appropriate learning outcomes or 
test objectives and in selecting specific test items to be included on the 
Grade 6 Connecticut Mastery Test (CMT). To have been considered for use, test 
objectives and Items must have been: 

(1) significant and important; 

(2) developmentally appropriate; 

(3) reasonable for most students to achieve; and 

(4) generally representative of what is taught in Connecticut schools. 

Once the objectives were Identified, item specifications and/or sample items 
were written. Item specifications are written descriptions of the types and 
forms of test items that assess an objective. They also prescribe the types 
of answer choices that can be used with each item. 

After the test specifications were written and agreed upon, the test 
contractor wrote items and response choices for each of the objectives. The 
items were then reviewed by the content committees. Items which met the 
criteria of the test specifications and received the approval of the content 
committees were considered for the pilot test. Before testing, the Bias 
Advisory Committee reviewed each item for potential discrimination related to 
gender, race or ethnicity in the language or format of the question or 
response choices. After their review was completed, the pilot test forms were 
constructed. Over 1,600 customized Connecticut items were included in the 
October 1985 grade six pilot test in language arts and mathematics. 
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The Psychometrics Advisory Committee provided advice concerning other aspects 
of the pilot test including the sampling design, statistical bias analysis, 
the design of item specifications and pilot test administration procedures. 
The recommendations proposed by the Psychometrics Advisory Committee were 
reviewed and endorsed by the Connecticut Student Assessment Advisory Committee. 



Pilot Tests 

After the items had been reviewed, twelve test forms (six in mathematics and 
six in language arts) were piloted for the grade six test. The purpose of 
several pilot test forms was to ensure that enough test items were included to 
construct three comparable test forms from the pilot test results. 

Over 6,000 grade six students participated in the October 1985 pilot test. In 
January 1986, the pilot test results were made available to Connecticut State 
Department of Education (CSDE) staff. The process of selecting items to 
construct three comparable test forms began by the Bias Advisory Committee 
examining the pilot test statistics of each item for potential bias. As a 
result, some items were eliminated from the item pool. From the remaining 
items, test forms were constructed to be equivalent in content and difficulty 
at both the objective and total test levels. 

Once the items were sorted on this basis, the test contractor prepared three 
complete forms of the mathematics test and two complete forms of the language 
arts test. These forms were approved by the content committees- Each form 
was created to be equal in difficulty and test length. A third language arts 
test was constructed after a few additional items were piloted as part of a 
later test administration. Later, during subsequent CMT administrations, 
enough items were pilot tested to yield two additional test forms. The 
psychometric procedures used to construct each of these test forms focused 
primarily on the use of the one-parameter item response model. 



Survey 

In October 1985, a survey of preliminary grade six mastery test objectives was 
sent to over 4,000 Connecticut educators. The purpose of the survey was to 
determine (1) the importance of the proposed mathematics and reading/language 
arts objectives and (2) whether the objectives were taught prior to the 
beginning of grade six. Approximately a 45% response rate was achieved which 
included approximately one-third of the respondents representing urban school 
districts. Thirty-six of the original thirty-nine mathematics objectives were 
judged to be important learning skills. 
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Grade Six Mathematics Objectives 



The 36 objectives of the sixth-grade mathematics test are listed below. There 
are four test items for each objective • The number of items in each domain is 
indicated in the parentheses • 

CONCEPTUAL UNDERSTANDINGS (44) 

1. Order whole numbers less than one hundred thousand 

2. Identify the value of a digit in whole numbers less than one 
hundred thousand and rewrite whole numbers using expanded 
notation 

3. Rename whole numbers by regrouping 1,000's, 100* s, 10* s and I's 

4. Round whole numbers less than one hundred thousand to the 
nearest 1 ,000, 100 and 10 

5. Multiply and divide multiples of 10 and 100 by 10 and 100 

6. Identify equivalent fractions and mixed numbers using pictures 

7. Identify equivalent fractions and mixed numbers 

8. Convert between mixed numbers and improper fractions 

9. Identify decimals (.01 to 2.99) from pictorial representations 

10. Extend patterns involving numbers and attributes 

11. Identify an appropriate procedure for making estimates for whole 
number computations 

COMPUTATIONAL SKILLS (36) 

12. Add and subtract 2-, 3- and 4-digit whole numbers and money 
amounts less than $100.00 

13. Know multiplication and division facts 

14. Multiply 2- and 3-digit whole numbers and money amounts less 
than $100.00 by 1-digit numbers 

15. Divide 2- and 3-digit whole numbers by 1-digit numbers 

16. Add and subtract fractions and mixed numbers with like 
denominators (without regrouping mixed numbers) 

17. Find fractional parts of whole numbers 

18. Estimate sums and differences of whole numbers and money amounts 

19. Estimate products and quotients of whole numbers and money 
amounts (1-digit factor and 1-digit, whole number divisor) 

20. Estimate sums and differences of fractions and mixed numbers 

PROBLEM SOLVING/APPLICATIONS (44) 

21. Interpret graphs, tables and charts 

22. Identify the graph that best illustrates given data 

23. Identify number sentences from problems 

24. Solve 1-step problems involving whole numbers and money amounts 

25. Solve problems involving making change 

26. Solve 1-step problems involving fractions 

27. Solve 2-step problems involving whole numbers and money amounts 

28. Estimate a reasonable answer to a given problem 

29. Identify extraneous information in problems and solve problems 
with extraneous information 

30. Identify needed information in problem situations 

31. Solve process problems involving the organization of data 
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MEASUREMENT/GEOMETRY (20) 



32. Identify geometric figures 

33. Measure/determine perimeters and areas 

34. Estimate lengths and areas 

35. Select appropriate metric or customary units and measures 

36. Determine elapsed time 



Performance on all 36 math objectives is reported at the student, classroom, 
school, district and state levels. 
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Grade Six Language Arts Objectives 

There are eleven multiple-choice objectives and two holistic measures, one for 
reading and one for writing, within the sixth-grade language arts test. The 
number of items for each content area or objective is indicated in the 
parentheses • 



WRITING MECHANICS (40) 

1- Capitalization and Punctuation (12) 

1. Spelling (9) 

3- Agreement (15) 

4. Tone (4) 

STUDY SKILLS (16) 

5. Locating Information (11) 

5. Note-taking and Outlining (5) 

LISTENING COMPREHENSION (20) 
?• Literal (6) 

8. Inferential and Evaluative (14) 
READING COMPREHENSION (36) 

9- Literal (8) 

10. Inferential (14) 

11. Evaluative (14) 

DEGREES OF READING POWER (77) 

WRITING SAMPLE (1) 

Holistic scoring is provided for all students. Analytic scoring is 
provided for students who score at or below the remedial standard of 
4 (on a scale of 2-8). 



Performance on all eleven Language Arts objectives, the Degrees of Reading 
Power and the Writing Sample is reported at the student, classroom, school, 
district and state levels. 



APPENDIX D 

Remedial (Grant) Standard-Setting Process 

and 

Standard-Setting Committees 
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Remedial 



(Grant) Standard-Setting Process 



Background 

There are several acceptable strategies for setting standards on 
criterion-referenced tests. Each of the proposed methods has one or more 
unique characteristics. One common element to the various methods is that 
they all offer to the individuals who are setting the standards some process 
which reduces the arbitrariness of the resulting standard. Different methods 
accomplish this in different ways. All methods systematize the standard- 
setting process so that the result accurately reflects the collective informed 
judgment of those setting the standard. 



Types of Standard-Setting Methods 

Standard-setting methods can generally be categorized into three types: test 
question review, individual performance review and group performance review. 
Test question review methods specify a procedure for standard setters to 
examine each test question and make a judgment about that question. For 
example, standard setters might be asked to rate the difficulty or the 
importance of each question. These judgments are numerically scaled and then 
combined mathematically to produce a standard. Individual performance review 
methods also require standard setters to make judgments, but the judgments are 
made on the basis of examining data that indicate how well individual students 
perform on test items. These data may be based on actual pilot test results 
or projected results using mathematical theories. In this method, additional 
student information, such as grades, may also be used to inform the standard 
setters* Group performance review methods provide for judgments to be made 
based on the performance of a reference group of students. That is, standard 
setters review the group performance and make a determination where the 
standard should be set based on the group results. 



Selection of a Standard-Setting Method 

Several factors affect the choice of a particular standard-setting method. 
The type of test is one consideration. For example, some methods are only 
appropriate for multiple-choice questions or for single correct answer 
questions while other methods are more flexible. For instance, time 
constraints are a consideration if student performance data are necessary. 
In this case, a pilot test must be conducted and the test results must be 
analyzed prior to setting the standards. Another consideration is the 
relative importance of the decisions that will be made on the basis of the 
standard. For example, a classroom test affecting only a few students would 
not require as stringent a procedure as would a statewide test determining 
whether a student is allowed to graduate from high school. Other relevant 
factors include the number of test items, permanence of the standard, purpose 
of the test and the extent of available financial and other resources to 
support the standard-setting process. 
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On February 4, 1985. the Mastery Test Psychometrics Advisory Committee met to 
consider the issue of standard-setting procedures and voted unanimously to 
approve the following proposal. 



A PROPOSAL FOR SETTING THE REMEDIAL STANDARDS ON THE CONNECTICUT MASTERY TESTS 

!• Two standard-setting committees will be created: one for mathematics and 
one for reading and writing. 

2. This description of a minimally proficient student will be given to each 
of the committees: 



Imagine a student who is just proficient enough in reading, writing 
and mathematics to successfully participate in his/her regular 
sixth-grade coursework. 

3a. In mathematics, an adaptation of the Angoff procedure will be used. 

The committee will be provided with each item appearing on one form of the 
mathematics test. The committee will be given the following directions: 

Consider a group of 100 of these students who are just proficient 
enough to be successful in regular sixth-grade coursework. How many 
of them would be expected to correctly answer each of the questions? 

The committee will rate each item. The committee will then be given the 
opportunity to discuss their rating of each item. Sample pilot data will 
be presented. Committee members will be given the opportunity to adjust 
their item ratings. The item ratings will then be averaged in accordance 
with the Angoff procedure in order to produce a recommended test standard. 

b. In reading, the committee will review and discuss each passage of the 
Degrees of Reading Power (DRP) test. Student performance data will be 
presented. The committee will consider the reading difficulty that should 
be expected of a student at the grade level being tested. The committee 
members will identify the passage that has the appropriate level of 
reading difficulty consistent with the above description of a minimally 
proficient student. 

c. In writing, the committee will read four sample essays. These essays 
will have been prescored holistically (on a scale from 2 to 8) in order to 
rank the quality of the essays. Committee members will classify essays 
into one of three categories: 1) definitely NQI proficient, 2) borderline 
and 3) definitely proficient. These classifications will be discussed in 
light of the holistic scores. The committee will then classify 
approximately twenty-five additional essays. The essay ratings will be 
discussed in the same manner as the original four essays. When all essays 
have been discussed, the essays which fell in the borderline category will 
be focused upon to determine the standard. The committee will determine 
where, among the borderline essays, the standard should be established. 

4. The standards recommended in step 3 will be presented to the Connecticut 
Student Assessment Advisory Committee (formerly the Mastery Test 
Implementation Advisory Committee) for discussion and action. 
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Connecticut's Strategy 



Several steps were employed to create an acceptable and valid test standard 
for Connecticut tests* Initially, a separate standard-setting committee was 
convened for each test on which standards were to be set* Individuals were 
chosen to serve as members on the committee on the basis of their familiarity 
with the area being assessed and the nature of the examinees* One source of 
such members was the test content committees related to the project* For 
example, members of the Mathematics Advisory Committee were represented on the 
committee setting standards for the mathematics mastery test* 

The actual procedures used to set standards were an adaptation of a method 
proposed by William Angoff (1970)* This test question review method required 
members of a standard-setting committee to estimate the probability that a 
question would be correctly answered by examinees who possess no more than the 
minimally acceptable knowledge or skill in the areas being assessed* Standard 
setters then reviewed pilot test data for sample items as further evidence of 
the appropriateness of the judgments being made* The original probability 
estimates assigned to each test question were reviewed and adjustments made by 
the standard setters* The final individual item probabilities were summed to 
yield a suggested test standard for each member of the committee* The 
suggested standards were averaged across members of the committee to produce 
the recommended test standard* 

The recommended test standard was presented to the Connecticut Student 
Assessment Advisory Committee and the State Board of Education* 

In mid-March, Mathematics and Language Arts Standard-Setting Committees met to 
set the remedial standards for the Grade 6 Mastery Test* The following 
Information summarized the results of the standard-setting activities 
conducted by CSDE staff: 

I. Mathematics (144-item test) 

Using the procedures previously outlined, the standard setters rated each item 
and considered the pilot data* Committee members discussed items and were 
given the opportunity to adjust their initial ratings* The final ratings were 
averaged to produce a remedial standard* It was recommended that a raw score 
of 79 be the remedial mathematics standard* Below Is a summary of the ratings. 

Procedure # Judges Range t Mean X Correct Raw Score 

Angoff 20 35-62 55 79 

II* Reading (Degrees of Reading Power, 77-1 tern test) 

Standard setters used two procedures to establish a remedial reading 
standard* First, they examined the passages in the Degrees of Reading Power 
(DRP) test, asking themselves which passage is too difficult for the student 
who is just proficient enough to successfully participate in sixth-grade 
coursework* Discussion occurred throughout this selection process* 



Second, they examined textbooks which are typically used in grades five and 
six and selected those textbooks which a minimally proficient student would 
not be expected to read in order to successfully participate in sixth-grade 
coursework* Discussion occurred throughout this selection process* 

The average readability values of the selected passages and textbooks and the 
pilot test data were then revealed to the standard setters. The standard 
setters discussed the readability values and the pilot test data and 
recommended the DRP unit score of 50 as the remedial standard* This standard 
was accepted by the State Board of Education at the 75% comprehension level. 
Below is a summary of the ratings* 

Readabi 1 i ty Recommended 
Procedure # Judges Range Remedial Standard 

A. Test Passage Review 25 49-56 DRP Units 

50 DRP Units 

B. Textbook Review 25 47-59 DRP Units 



III* Writing (45-minute writing sample) 

Using the procedure previously outlined, standard setters read and rated 21 
essays written to a narrative prompt and 21 essays written to an expository 
prompt. After discussions and final ratings, the holistic scores for the 
papers were revealed to the group. The committee then discussed the 
appropriate remedial writing standard in light of the degree to which their 
ratings matched the holistic scores. It was the recommendation of the 
committee that a holistic writing score of 4 be used as the remedial writing 
standard. Below is a summary of the ratings. 



NARRATIVE PROMPT 



Rating After Discussion 



Holistic 


Definitely 




Definitely 


Score 


NOT Proficient 


Borderl ine 


Proficient 


2 


100% 


0% 


0% 


3 


72% 


0% 


28% 


4 


9% 


0% 


91% 


5 


0% 


0% 


100% 


6 


4% 


0% 


96% 


7 


1% 


0% 


?9% 


8 


0% 


0% 


100% 


EXPOSITORY PROMPT 


Ratina After Discussion 


Holistic 


Definitely 




Definitely 


Score 


NOT Proficient 


Borderl ine 


Proficient 


2 


100% 


0% 


0% 


3 


100% 


0% 


0% 


4 


16% 


0% 


84% 


5 


6% 


0% 


94% 


6 


0% 


0% 


100% 


7 


0% 


0% 


100% 


8 


0% 


0% 


100% 
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Standard-Setting 
LANGUAGE ARTS STANDARD-SETTING COMMITTEE 



Commi ttees 



Cheryl Anderson, Thompson Public Schools 

Roberta Bellows, Trumbull Public Schools 

Joseph Bibbo, Stonington Public Schools 

Dell Britt, Newtown Public Schools 

Eileen Brunt, Region School District No, 7 

Evelyn Burnham, Region School District No- 7 

Dorothy French, Litchfield Public Schools 

Marguerite Fuller, Bridgeport Public Schools 

Nina Grecenko, Newtown Public Schools 

John Hennelly, Old Saybrook Public Schools 

David Johnson, Thompson Public Schools 

Robert Kinder, CT State Department of Education 

Angela Kiss, Windham Public Schools 

Jean Klein, Newtown Public Schools 

Christopher Kotsaftis, Litchfield Public Schools 

Addie Lindsey, Bridgeport Public Schools 

Ethan Margolis, Stamford Public Schools 

Dick Nelson, Old Saybrook Public Schools 

Bruce Olean, Stonington Public Schools 

Anne Stasiewski, Norwalk Public Schools 

Marcia Van Hise, Trumbull Public Schools 

Deborah Wallerstein, Norwalk Public Schools 

Susan Webb, Windham Public Schools 

Mary Wei nl and, CT State Department of Education 

Mary Wilson, Hartford Public Schools 



MATHEMATICS STANDARD-SETTING COMMITTEE 

Pat Banning, Windham Public Schools 

Barbara Bioty, Windham Public Schools 

Betsy Carter, CT State Department of Education 

Mitchell Chester, Farmington Public Schools 

Jo Anne Davidson, Westport Public Schools 

Coretta Dean, Bridgeport Public Schools 

Karol DeFalco, New Haven Public Schools 

Robert Dingee, Norwalk Public Schools 

Ralph Esposito, New Haven Public Schools 

Steve Leinwand, CT St?*-e Department of Education 

Peter Lovely, Bloomfield Public Schools 

Ellen Morse, Manchester Public Schools 

John O'Neal, Farmington Public Schools 

Marilyn Parker, Manchester Public Schools 

Scarlett Pipkin, Bridgeport Public Schools 

Arlene Schaffer, Ashford Public Schools 

Jo Shay, Westport Public Schools 

Martha Strickland, Middletown Public Schools 

Sylvia Webb, Middletown Public Schools 

Joan Webster, Norwalk Public Schools 
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APPENDIX E 
Grade Six Overview of Holistic Scoring 
and 

Marker Papers for Holistic Scoring 
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An Overview of Holistic Scoring 



Description of the Method 

Holistic scoring involves judging a writing sample for Its total effect. 
The scorer makes an overall evaluation taking into account all characteristics 
which distinguish good writing. No one feature (such as spelling, rhetoric, 
or organization) should be weighted to the exclusion of all other features. 
Contributing to the rationale underlying holistic scoring is evidence that: 

0 no aspect of writing can be judged independently and result in an 
overall score of quality; 

0 teachers can recognize and concur upon good writing samples; and 

0 teachers tend to rank entire pieces of writing in the same way, 
regardless of the importance they might attach to the particular 
components of writing. 

The scoring scale for holistic scoring Is determined by the quality of the 
specific samples being evaluated. That is, the success of a particular 
response is determined in relationship to the range of ability reflected in 
the set of writing samples being assessed. 

Preparation for Scoring 

Prior to the training/scoring sessions, a committee consisting of Connecticut 
State Department of Education (CSDE) consultants, representatives of the 
Language Arts Advisory Committee and other language arts specialists from 
throughout the state, two chief readers and a project director from 
Measurement Inc. of Durham, North Carolina, and a reading specialist from The 
Psychological Corporation met and read a substantial number of essays drawn 
from the total pool of essays to be scored. Approximately 60 essays were 
selected to serve as "range-finders" or "marker papers" representing the range 
of achievement demonstrated in the total set of papers. Copies of those 
range-finders served as training papers during the scoring workshops which 
followed. Each range-finder paper was assigned a score according to a 
four-point scale, where 1 represented a poor paper and 4 represented a 
superior paper. 

Scoring Workshops 

During the month of November, several holistic scoring workshops were held in 
various locations throughout the state. Attendance at the grade six scoring 
workshops totaled 266 teachers. A chief reader and two assistants were 
present at every workshop in addition to representatives of the CSDE. Each 
workshop consisted of a training session and a scoring session. 

Training and Qualifying 

0 All teachers were shown approximately fourteen range-^finder papers. 
The chief reader discussed each paper and explained the reason why 
each received its score. 
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0 All teachers were given a six-paper practice set. They scored the 

papers independently and recorded the scores on their papers. When 

all teachers were finished, the chief reader discussed each paper and 
explained why each received its score* 

0 All teachers were given a nine-paper training set. They scored the 
papers independently, based on an overall impression, and recorded 
their scores on a monitor sheet as well as on their papers. As they 
finished reading and scoring, they brought the monitor sheet to the 
team leader who checked the scores. When all teachers were finished 
and all monitor sheets were checked, the chief reader discussed the 
nine-paper set, 

0 Regardless of whether or not they qualified on the first training 
set, all teachers were then given another nine-paper training set. 
They scored the papers and had the monitor sheets checked. Set Two 
was not discussed, except with non-qualifiers, 

0 Teachers were considered qualified if they scored six or more papers 
correctly on either set. Teachers who met the standard began scoring 
actual test papers after Set Two, 

0 If any teacher did not qualify, they received additional training by 
one of the team leaders or by the chief reader away from the scoring 
room. They had two more opportunities to qualify. Any teacher who 
failed to qualify would have been excused from the project and paid 
for one day. 

The Scoring Session 

Once scorers qualified, actual scoring of the writing exercises began 
according to the steps outlined below: 

0 Scorers read each paper once carefully but quickly and designated a 
score. Again, the score reflected the scorer's overall impression of 
the response as it corresponded with the features of written 
composition which were internalized during the training process, 

0 Each paper was read and scored by a second scorer independently of 
the first, that is, without seeing the score assigned by the first 
reader. The chief reader had the responsibility of adjudicating any 
disagreement of more than one point between the judgments of the 
first two scorers. In other words, adjacent scores (i,e,, awarded 
scores of 4 and 3, 1 and 2, 2 and 3) were acceptable, but larger 
discrepancies (i,e,, scores of 2 and 4, 3 and 1, 1 and 4) were 
resolved by the chief reader. In general, with successful training, 
the occurrence of large score discrepancies is rare. 



0 



The two scores for each paper were added to produce the final score 
for each student, resulting in scores between 2 and 8, 



Understanding the Holistic Scores 



Examples of actual student papers which are representative of the scoring 
range will assist the reader in understanding the statewide standard set for 
writing and Interpreting the test results. Sample papers representing four 
different holistic scores are presented on the following pages* Note that the 
process of summing the scores assigned by the two readers expands the scoring 
scale to account for "borderline" papers* A paper which receives a 4 from 
both scorers (for a total score of 8) is likely to be better than a paper to 
which one reader assigns a 4 and another reader assigns a 3 (for a total score 
of ?)• In addition. It should be emphasized that each of the score points 
represents a range of student papers—some 4 papers are better than others. 

A score of Not Scorable (NS) was assigned to student papers in certain cases. 
A score of NS Indicates that the student *s writing skills remain to be 
assessed. The cases in which a score of NS was assigned were as follows: 

0 responses merely repeated the assignment; 

0 Illegible responses; 

0 responses In languages other than English; 

0 responses that failed to address the assigned topic in any way; and 

0 responses that were too brief to score accurately, but which 

demonstrated no signs of serious writing problems (for example, a 
response by a student who wrote the essay first on scratch paper and 
who failed to get very much of It copied). 

Both readers had to agree that a paper deserved a NS before this score was 
assigned. If the two readers disagreed, the chief reader arbitrated the 
discrepancy. Papers which were assigned a score of NS were not included in 
summary reports of test results. 

Summary Comments 

The fact that standards must be maintained and reinforced throughout a scoring 
session cannot be overemphasized. Holistic scoring depends for its usefulness 
on consistency of scoring among all scorers throughout the sessions. 
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CONNECTICUT MASTERY TEST 
1991 Grade Six 
Writing Assignment 



Suppose that your parents or guardians asked you how you would like to spend 
your summer. What would you choose? Why would you choose this? Explain to 
your parents or guardians why you chose what you did. 

• Choose ONE way you would like to spend your summer. 

• Decide why you would like to do this. 

• Write to your parents or guardians explaining why you want to spend 
your summer doing this. 
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APPENDIX F 
Grade Six Analytic Rating Guide 
and 

Marker Papers for Analytic Scoring 



Grade Six Analytic Rating Guide 
FOCUS: How effectively does the writer unify the paper by a dominant topic? 

1 - switches and/or drifts frequently from the dominant topic 

2 « switches and/or drifts somewhat from the dominant topic 

3 « stays on topic throughout the response 

ORGANIZATION: Is there a plan that clearly governs the sequence from the 
beginning to the end of the response, and is the plan effectively signaled? 

1 « no discernible plan 

2 « inferable plan and/or discernible sequence; some signals may be 

present 

3 « controlled, logical sequence with a clear plan 

SUPPORT/ ELABORATION: To what extent is the narrative developed by details 
that describe and explain the narrative elements (character, action and 
setti ng)? 

1 « vague or sketchy details that add little to the clarity of the 

response or specific details but too few to be called list-like 

2 « details that are clear and specific but are list-like, or uneven, or 

not developed 

3 « somewhat^developed details that enhance the clarity of the response 

CONVENTIONS: To what extent does the student use the conventions of 
standard written English (e.g., sentence formation, spelling, usage, 
capitalization, punctuation)? 

1 « many errors 

2 « some errors 

3 m few errors 
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APPENDIX G 

Sample Grade Six Mastery Test Score Reports 

0 Class Diagnostic Report 

- Mathematics 

0 School by Class Report 

- Mathematics 

0 District by School Report 

- Mathematics 

0 Class Diagnostic Report 

- Language Arts 

0 School by Class Report 

- Language Arts 

0 District by School Report 

- Language Arts 

0 Parent/Student Diagnostic Report 
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APPENDIX I 
Fall 1991 Grade Six 
State by District Report: 
Language Arts 
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Type of Community 



TOC 1 - LARGE CITY - a town wUh a population of more than 100,000. 

TOC 2 - FRINGE CITY - a town contiguous with a large city and with a 
population over 10,000. 

TOC 3 . MEDIUM CITY - a town with a population between 25,000 and 100,000 and 
not a Fringe City. 

TOC 4 - SMALL TOWN (Suburban) - a town withir^ an SMSA* with a population of 
less than 25,000, not a Fringe City. 

TOC 5 - SMALL TOWN (Emerging Suburban) - a town with a population of less than 
25,000 included in what was a proposed 1980 SMSA but not included in a 
1970 SMSA. 

TOC 6 - SMALL TOWN (Rural) - a town not included in an SMSA, with a population 
of less than 25,000. 



*Standard Metropolitan Statistical Area 
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Education Reference Group Descriptions 



The education reference groups were formed from an analysis of districts* 
median family income, a percentage of high school graduates, a percentage of 
those in managerial /professional occupations, a percentage of single-parent 
families, a percentage of those below poverty and a percentage of non-English 
home language from the 1980 census. The groups have not been named, but have 
been labeled I through VII. Note, however, that the groups run from extremely 
affluent suburban communities (I) to our three largest cities of Hartford, 
Bridgeport and New Haven (VII). Some differ widely with respect to all of the 
family background variables; others differ slightly with respect to one or 
two. In addition to the six variables used to classify districts, the group 
descriptions below also include superintendents' comments that were provided 
in a Department survey in 1988. 

Group I. These 13 districts were wealthy, professional suburbs. The median 
family Income in 1979 averaged $40,425. Residents were extremely well 
educated. Nearly 90% had at least a high school diploma, 42% had a bachelor*s 
degree and 49X had a managerial or professional job. There were relatively 
few children with educational disadvantages here. Only 7X of the families 
were single-parent, about 8X spoke a language other than English at home and 
almost no one (2X) lived in poverty. Superintendents within these towns used 
the adjectives "suburban," "affluent," "growing" and "bedroom community" to 
describe them. 

Group II. Residents in the 29 districts of Group II were affluent, 
well-educated professionals, but to a lesser extent than residents of 
Group I. The median family Income averaged $28,113, more than 83X of the 
residents had high school diplomas, 29% had a college degree and 36% had a 
managerial or professional job. Like Group I, this group had a low percentage 
of people who spoke another language at home (8X), almost no one in poverty 
(2X) and relatively few single-parent families (9%). Like the superintendents 
in Group I, superintendents from these towns described their communities as 
"affluent," "bedroom communities," "growing" and "suburban." 

Group III. These 34 districts were mostly rural bedroom communities. Like 
Groups I and II, these towns did not have many disadvantaged children. There 
were only 1% who spoke a language other than English at home, only 7X who were 
from single-parent families and only 3% who were poor. Adults were slightly 
less affluent (median family Income of $24,431), less likely to have a high 
school diploma (77%) and less likely to have a managerial or professional job 
(28%) than people in Group II. Like the previous two groups, these towns were 
described by superintendents as "suburban," "growing" and "bedroom 
communities." Several superintendents used "rural" and "middle class" (as 
well as "affluent") to describe their communities. 
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Group IV. This group of 37 districts was probably the most diverse set of 
towns » containing a number of coastal and resort communities, as well as rural 
and suburban areas- Group IV was similiar to Group III in median family 
income ($22,609), percentage of high school graduates (77%), percentage of 
managers/professionals (29%) and percentage of non-English home language (7%), 
but had a significantly higher percentage of single-parent families 
(12% versus 7%) and a slightly higher percentage of families below poverty 
(5X versus 3t). Superintendents* descriptions reflect this group's 
diversity. They describe their towns as "bedroom," "growing," "rural," 
"suburban," "middle income" and "affluent." 

Group V. These 30 districts made up the first group of working class/blue 
collar communities. This group had a significantly lower percentage of high 
school graduates (68%) and percentage of managers/professionals (19%) than 
Group IV. Other characteristics were similar to Group IV: the average income 
was $21,920, there were 11% single-parent families, 5% below poverty and 9% of 
the population spoke a language other than English at home. 

Group VI. This group of 23 districts included the state's medium-sized 
cities, the larger cities of Stamford and Waterbury, sever?" former mill towns 
and some densely populated blue collar suburbs. Group VI had similar 
socioeconomic characteristics as Group V, but significantly greater 
proportions of single-parent families and families in which English was not 
the primary home language. The median family income of $20,325 was below the 
state average. An average of 16% of the residents spoke another language at 
home and 17% of the families were headed by single parents. Only 63% of the 
residents had high school diplomas, and 6% lived below poverty level. 

Group VII. Hartford, Bridgeport and New Haven were vastly different from 
other communities in Connecticut. An average of 28% of the families spoke a 
language other than English, 46% were headed by single parents, 20% lived in 
poverty and the median family income was $15,240. 
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APPENDIX L 
Student Participation Rates 
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Connecticut State 
Department of Education 



Division of Research, Evaluation and Student Assessment 

Douglas A. Rindone, Acting Director 

Bureau of Evaluation and Student Assessment 

Peter Behuniak, Acting Chief 

Student Assessment and Testing Unit 

William J. Congcro, Coordinator 

Patricia Brandt 
Richard Cloud 
Naomi Wise 
Education Service Specialists 

John B. Rogers 
Education Service Assistant 

Richard E Mooney, Senior Program Associate 

Stephen Martin, Assessment Associate 

Division of Curriculum and 
Professional Development 

Betty Sternberg, Director 

Steven Leinwand 
Mari Muri 
Mathematics Consultants 

Karen Costello 
Reading/Language Arts Consultant 



It is the policy of the Connecticut State Board of Education that no person shall be excluded 
from participation in, denied the benefits of, or otherwise discriminated against under any 
program, inckiding employment, because of race, color, religion, sex, age, national origin, 
ancestry, marital status, mental retardation, mental or physical disability, including, but not 
limited to blindness. 
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